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Present  Status 


During  the  past  five  months  work  has  been  concentrated  on  four  topics, 
namely  (1)  further  elaboration  of  the  conceptual  scheme  for  the  automatic  ex¬ 
traction  of  the  traffic  network  from  aerial  images,  (2)  production  of  results  at 
the  iconic  image  analysis  level,  (3)  the  implementation  of  two  segmentation 
procedures,  (4)  and  investigation  of  a  blackboard-oriented  production  system 
for  image  understanding  (BPI)  in  the  context  of  traffic  network  extraction. 


1  Conceptual  Scheme 


■c?  Investigations  of  three  different  methods  for  the  automatic  extraction  of  the 
traffic  network  from  aerial  imagery  of  rural  as  well  as  urban  areas  have  shown 
the  strengths  and  weaknesses  of  these  methods. 

A  method  for  the  automatic  extraction  of  line  objects  from  large  aerial  im¬ 
ages  of  prevailing  rural  areas  [4^2} 'Vvorks  successfully  in  this  domain.  However, 
when  applied  to  images  of  urban  areas,  success  and  reliability  of  the  method 
decrease  remarkably  due  to  the  complex  mixture  of  natural  and  man  made 
objects  at  those  locations.  It  seems  to  be  impossible  to  solve  this  problem 
through  specialized  signal  processing  methods  without  incorporation  of  seman¬ 
tics,  i.e.  detailed  modelling  of  prototypical  objects  and  object  arrangements  to 
represent  housing  areas,  plant  areas,  traffic  areas,  etc. 

-*  A  next  method  we  have  examined  is  best  described  as  knowledge-based  ap¬ 
proach  for  the  analysis  of  aerial  images  f3j!  In  this  approach  image  analysis 
is  performed  at  three  levels  of  abstraction,  namely  iconic  or  low-level  image 
analysis,  symbolic  or  medium-level  image  analysis,  and  semantic  or  high-level 
image  analysis.  Domain  dependent  knowledge  about  prototypical  urban  areas 
is  incorporated  via  a  semantic  network.  /  The  method  works  well  in  urban  ar¬ 
eas  that  are  not  too  complex,  like  tha'  aerial  image  of  Phoenix,  Arizona.  It 
can  be  anticipated  that  a  more  elaborated  control  mechanism  combined  with 
enhanced  segmentation  methods  enable  the  system  to  process  successfully  even 
more  complex  imagesr-jHowever,  depart  from  using  enhanced  segmentation,  we 
decided  not  to  follow  this  approach,  because  a  third  method  we  have  investi¬ 
gated  will  promise  reasonable  results  at  lower  cost. 


This  method  uses  a  blackboard-oriented  production  system  for  image  un¬ 
derstanding  (4,5).  Under  the  premise  that  the  crossings  of  the  traffic  network 
can  be  detected,  this  method  is  able  to  extract  the  traffic  network  from  aerial 
images  of  a  complexity  like  the  Bietigheim  image  (South-West  Germany). 

Our  approach  for  the  automatic  extraction  of  the  traffic  network  from 
aerial  images  consists  of  a  combination  of  the  three  methods  described  above:’ 

•  Application  of  the  signal  processing  method  for  road  extraction  in  rural, 
areas  to  get  cues  for  the  extraction  of  arterial  roads  at  the  boundaries 
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from  rural  to  urban  areas. 


•  Enhancement  of  the  blackboard-oriented  approach,  using  segmentation 
methods  developed  for  the  knowledge-based  approach. 

•  Application  of  the  enhanced  blackboard-oriented  approach  to  extract  the 
traffic  network  within  urban  areas  using  the  cues  of  arterial  roads  and 
crossings. 

In  the  following  we  describe  the  iconic  and  symbolic  image  analysis  levels 
carried  out  up  to  now  (Fig.  1). 

First,  we  compute  from  the  digitized  image  a  gray-level  multi-resolution 
pyramid.  Then  we  apply  a  spot-detector  as  well  as  an  edge-detector  to  the 
gray-level  pyramid,  computing  spots  resp.  edge  elements.  The  original  idea  to 
implement  also  a  line-detector  has  been  modified;  it  is  a  significant  improve¬ 
ment  to  detect  lines  at  a  later  stage  of  image  analysis,  e.g.  as  adjacent  anti- 
parallel  edges. 

At  the  symbolic  image  analysis  level  two  segmentation  procedures  are  at 
hand,  computing  area-  resp.  edge-like  image  segments. 

Using  the  blackboard-oriented  environment,  image  analysis  proceeds  by  de¬ 
tecting  regular  arrangements  and  structures  between  image  segments  and  im¬ 
age  structures.  At  a  first  stage  simple  image  structures  are  detected,  e.g. 
by  merging  adjacent  anti-parallel  edges  to  lines  or  by  merging  short  edges  to 
longer  ones,  obeying  the  Gestalt  principles  of  symmetry  resp.  good  continu¬ 
ation.  At  a  later  stage  of  analysis  interdependencies  between  structures  are 
detected,  e.g.  parallelisms  between  neighboring  lines.  For  that  purpose  we 
introduce  two  other  sections  in  the  conceptual  scheme  entitled  ” simple  image 
structures**  resp.  ’’composite  image  structures**. 

2  Results  of  Spot-  and  Edge-Detection 

As  we  have  pointed  out  in  our  last  report,  low-level  image  analysis  is  done  by 
applying  competing  knowledge-sources  or  experts.  In  particular  we  have  imple¬ 
mented  a  spot-detector,  and  an  edge-detector. 

Figure  2  shows  the  gray-level  pyramid  of  an  urban  scene.  Figure  3  (upper 
part)  shows  the  strength  response  of  the  spot-detector  applied  to  the  gray-level 
pyramid  of  figure  2.  Fig.  3  (lower  part)  shows  the  surroundedness  response 
of  the  spot-detector  applied  to  the  same  scene.  Strength  response  as  well  as 
surroundedness  response  are  used  by  an  appropriate  segmentation  procedure  to 
obtain  area-like  segments  of  the  input  image. 

To  obtain  edge-like  segments,  the  response  of  an  edge-detector  is  used.  In 
our  case  we  use  the  Sobel  operator  to  detect  edges.  Nevertheless  any  edge- 
detector  computing  strength  and  direction  responses  of  an  edge  can  be  used. 
Figure  4  (above)  shows  the  strength  response  of  the  Sobel  operator  applied  to 
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Figure  2:  Gray-level  pyramid  of  an  urban  scene 


figure  2,  whereas  the  lower  part  of  figure  4  shows  the  direction  response  of  the 
operator  applied  to  the  input  image  given  in  figure  2  (the  direction  values  are 
coded  as  gray-levels). 


3  Segmentation 


3.1  Area-Oriented  Segmentation 

Dark  or  bright  contrasting  area-like  segments  are  obtained  by  applying  the 
area-oriented  segmentation  procedure.  Input  to  the  segmentation  procedure 
are  single  dark  or  bright  contrasting  spots  or  sets  of  spots. 

Segmentation  is  done  by  the  following  algorithm: 


1.  Select  a  dark/bright  contrasting  spot  in  level  n  of  the  spot-pyramid, 
so  that  strength  and  surroundedness  responses  of  the  selected  spot  are 
greater  than  a  given  threshold. 

2.  Project  the  selected  spot  from  level  n  of  the  spot-pyramid  to  level  n  +  i 
of  the  gray-level  pyramid. 

3.  Compute  the  meanvalue  of  the  gray-level  image  within  the  projection 
area  of  the  enlarged  spot. 

4.  Take  the  computed  meanvalue  as  a  threshold  and  binarize  the  gray-level 
image  within  the  patch  (the  Na  neighborhood  of  the  enlarged  spot). 
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Figure  4:  Edge-detector,  strength  (upper  part),  and  direction  (lower  part) 
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Figure  5:  Area-oriented  segmentation  of  bright  contrasting  spots 

5.  Compute  additional  features  of  the  connectivity  components  of  the 
patch. 

6.  Compute  confidence  values  of  the  feature  vectors  of  the  connectivity 
components. 

7.  Threshold  the  confidence  values  of  the  feature  vectors.  Reject  connectiv¬ 
ity  components  with  confidence  values  less  than  a  given  threshold.  Ac¬ 
cept  components  with  confidence  values  greater  or  equal  than  a  given 
threshold. 

8.  Store  accepted  components  in  the  blackboard. 

Figure  5  shows  segmentation  results  received  by  projecting  bright  spots 
from  level  3  of  the  spot-pyramid  to  levels  4, . . . ,  8  of  the  gray-level  pyramid 
and  applying  the  segmentation  algorithm  described  above.  Another  example  of 
applying  the  segmentation  procedure  is  given  in  figure  6.  Here  the  segments 
are  obtained  by  projecting  dark  spots  from  level  3  of  the  spot-pyramid  to  lev¬ 
els  4, ...  ,8  of  the  gray-level  pyramid. 

In  the  first  case  most  of  the  bright  contrasting  houses  are  detected  by  the 
segmentation  procedure.  The  segmentation  of  dark  spots  leads  to  the  detection 
of  dark  contrasting  segments,  containing  a  great  portion  of  the  traffic  network 
of  the  input  image. 


Figure  6:  Area-oriented  segmentation  of  dark  contrasting  spots 


3.2  Edge-Oriented  Segmentation 

Edge-like  segments  are  obtained  by  applying  the  edge-oriented  segmentation 
procedure.  Input  to  the  segmentation  procedure  are  edge  elements  computed 
by  the  Sobel  operator. 

Segmentation  is  done  by  the  following  algorithm: 

1.  Select  an  edge  element  in  level  n  of  the  edge- pyramid,  so  that  the 
strength  response  of  the  edge  element  is  greater  than  a  given  threshold. 
This  edge  element  is  treated  as  the  start  element  of  the  edge. 

2.  Look  for  adjacent  predecessor/successor  elements  of  the  edge. 

3.  Test  compatibility  of  the  predecessor/successor  elements  with  the  current 
edge  model. 

4.  Extend  the  edge  with  the  most  compatible  predecessor /successor  ele¬ 
ment. 

5.  If  no  compatible  predecessor /successor  element  can  be  found,  go  to 
step  8. 

6.  Update  the  edge  model. 

7.  Go  to  step  2. 

8.  Compute  additional  features  of  the  edge. 
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Figure  7:  Edge-oriented  segmentation 


9.  Compute  confidence  values  of  the  feature  vector  of  the  edge. 

10.  Threshold  the  confidence  values  of  the  feature  vector.  Reject  edges  with 
confidence  values  less  than  a  given  threshold.  Accept  edges  with  confi¬ 
dence  values  greater  or  equal  than  a  given  threshold. 

11.  Store  accepted  edges  in  the  blackboard. 

Figure  7  shows  segmentation  results  received  by  applying  the  segmentation 
procedure  described  above  to  levels  4, ...  ,8  of  the  gray-level  pyramid.  Most  of 
the  principal  edges  are  detected  by  the  edge-oriented  segmentation  procedure. 

4  Blackboard- Oriented  Production  System 

For  analysing  objects  in  images  one  can  use  syntactical  classificators,  which 
associate  each  object  with  a  special  class  of  account  by  analysing  the  struc¬ 
ture  of  these  image-objects.  A  special  realization  of  that  classificator  is  the 
blackboard-oriented  production  system  for  image  understanding  (BPI)  imple¬ 
mented  at  FIM  [4,5]. 

Starting  with  low-level  elements  (terminals)  figure  8a,  generated  by  picture 
preprocessing,  the  system  generates  more  and  more  complex  objects  (nonter- 
minales)  up  to  the  goal-object.  To  build  more  complex  objects  from  less  com¬ 
plex  ones,  the  system  has  to  test  hypotheses  about  the  objects.  Each  hypoth¬ 
esis  is  tested  by  a  knowledge  source  of  the  blackboard-system.  The  generated 
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a) 


b) 


Figure  8:  a)  image  with  low-level  elements,  b)  result  from  the  classification  of 
crossings 

objects  are  stored  in  the  blackboard  memory  for  associative  access.  All  knowl¬ 
edge  sources  are  used  in  parallel  to  get  all  possible  interpretations  of  the  im¬ 
age.  One  object  can  be  part  of  several  more  complex  objects. 

Figure  8a  shows  the  input-image  with  the  low-level  elements  included.  In 
figure  8b  results  from  the  classification  of  crossings  are  shown. 


Continuation  of  Work 

Next  topic  to  be  done  is  to  integrate  existing  low-level  and  medium-level  algo¬ 
rithms  into  the  blackboard-oriented  environment.  It  is  also  necessary  to  test 
the  performance  of  the  blackboard-oriented  approach  with  other  images.  Then 
the  final  phase  of  the  contracted  work  should  dedicated  to  the  exploratory  of 
the  traffic  network  extraction  in  urban  areas. 
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